Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding fully asynchronous versions of connect and publish. #1311

Merged
merged 7 commits into from
May 2, 2023

Conversation

stebet
Copy link
Contributor

@stebet stebet commented Mar 14, 2023

Proposed Changes

Now that the library is using Pipelines we can create fully asynchronous versions of the RabbitMQ operations. Here is a draft PR that implements async methods to run through the connection handshake and makes an async version of BasicPublish.

TODO:

  • Run comparison benchmarks
  • Documentation
  • More tests

Types of Changes

  • Bug fix (non-breaking change which fixes issue #NNNN)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause an observable behavior change in existing systems)
  • Documentation improvements (corrections, new content, etc)
  • Cosmetic change (whitespace, formatting, etc)

Checklist

  • I have read the CONTRIBUTING.md document
  • I have signed the CA (see https://cla.pivotal.io/sign/rabbitmq)
  • All tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • Any dependent changes have been merged and published in related repositories

Further Comments

Going to make this a draft for now. If this makes sense, we can gradually move other operations to be async as well, and hopefully even remove the synchronous versions completely later on.

@lukebakken lukebakken self-assigned this Mar 14, 2023
@lukebakken lukebakken added this to the 7.0.0 milestone Mar 14, 2023
@lukebakken lukebakken marked this pull request as ready for review March 14, 2023 16:37
@lukebakken
Copy link
Contributor

Thanks @stebet! I don't want to have pull requests staying open for a long time so I'm going to fix conflicts here, review, and get it merged ASAP!

@PauloHMattos
Copy link

@stebet with this change will be possible to avoid the copy in BasicPublish as described here?

@stebet
Copy link
Contributor Author

stebet commented Mar 14, 2023

@stebet with this change will be possible to avoid the copy in BasicPublish as described here?

I think so yes. Let me read over this again tomorrow to make sure.

@stebet
Copy link
Contributor Author

stebet commented Mar 15, 2023

@stebet with this change will be possible to avoid the copy in BasicPublish as described here?

Yes, this PR solves this issue where memory usage goes up the roof if the producing app is sending messages faster than they can be delivered over the network, since the output buffer is no longer unlimited. It is now limited to 128 messages and if that is full, the BasicPublish method blocks until more space is available in the buffer.

@lukebakken lukebakken force-pushed the asyncconnectandpublish branch from 5ca18ac to 7059d13 Compare March 24, 2023 15:42
@lukebakken lukebakken force-pushed the asyncconnectandpublish branch from 7059d13 to 63985eb Compare April 12, 2023 22:42
@lukebakken
Copy link
Contributor

I'm trying to suss out the CI build failures. Works on my machine, of course.

@stebet
Copy link
Contributor Author

stebet commented May 2, 2023

Anything left to resolve here @lukebakken?

@lukebakken
Copy link
Contributor

@stebet nope, just waiting around to see if anyone else will review. I just got back from some time off. Merging it!

@lukebakken lukebakken merged commit 5b64ca9 into rabbitmq:main May 2, 2023
@lukebakken lukebakken deleted the asyncconnectandpublish branch May 2, 2023 13:50
@lukebakken
Copy link
Contributor

@stebet let me know what you plan to "async-ify" next (if anything, you've contributed so much so far) and I will work on another section of the code.

Copy link

@Kuinox Kuinox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,
Are you really sure about that?
It can cause deadlocks in some scenarios:
https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md#warning-deadlocks

var task = _frameHandler.WriteAsync(memory);
if (!task.IsCompletedSuccessfully)
{
task.AsTask().GetAwaiter().GetResult();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, are you really sure about that?
It can cause deadlocks in some scenarios:
https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md#warning-deadlocks

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would appreciate a pull request with your ideas for improvements. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants